Problem statement (Term Deposit Sale)¶
Goal¶
Using the data collected from existing customers, build a model that will help the marketing team identify potential customers who are relatively more likely to subscribe term deposit and thus increase their hit ratio.
Resources Available¶
The historical data for this project is available in file https://archive.ics.uci.edu/ml/datasets/Bank+Marketing
Input variables:¶
Bank client data:
- age: Continuous feature
- job: Type of job (management, technician, entrepreneur, blue-collar, etc.)
- marital: marital status (married, single, divorced)
- education: education level (primary, secondary, tertiary)
- default: has credit in default?
- housing: has housing loan?
- loan: has personal loan?
- balance in account Related to previous contact:
- contact: contact communication type
- month: last contact month of year
- day: last contact day of the month
- duration: last contact duration, in seconds* Other attributes:
- campaign: number of contacts performed during this campaign and for this client
- pdays: number of days that passed by after the client was last contacted from a previous campaign (-1 tells us the person has not been contacted or contact period is beyond 900 days)
- previous: number of contacts performed before this campaign and for this client
- poutcome: outcome of the previous marketing campaign Output variable (desired target):
- Target: Tell us has the client subscribed a term deposit. (Yes, No)
In [1]:
#Import all the necessary modules
import warnings
warnings.filterwarnings('ignore')
#Load Libraries
#!pip install pandas_profiling
import pandas as pd
import pandas_profiling
import numpy as np
import seaborn as sns
from sklearn import preprocessing
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.feature_selection import RFE
from sklearn import metrics
from sklearn.metrics import classification_report, confusion_matrix,recall_score, precision_score, f1_score, roc_auc_score,accuracy_score
import matplotlib.pyplot as plt
%matplotlib inline
# Remove scientific notations and display numbers with 2 decimal points instead
pd.options.display.float_format = '{:,.2f}'.format
num_bins = 10
In [2]:
bank_df = pd.read_csv('bank-full.csv')
In [3]:
bank_df.head()
Out[3]:
In [4]:
bank_df.info()
In [5]:
bank_df.isna().sum()
Out[5]:
In [6]:
#size
bank_df.shape
Out[6]:
In [ ]:
Deliverable – 1 (Exploratory data quality report reflecting the following) – (20)¶
1. Univariate analysis (12 marks)¶
- Univariate analysis – data types and description of the independent attributes which should include (name, meaning, range of values observed, central values (mean and median), standard deviation and quartiles, analysis of the body of distributions / tails, missing values, outliers.
- Strategies to address the different data challenges such as data pollution, outlier’s treatment and missing values treatment.
- Please provide comments in jupyter notebook regarding the steps you take and insights drawn from the plots.
In [7]:
bank_df.describe().transpose()
Out[7]:
In [8]:
bank_df.columns.values
Out[8]:
In [9]:
#Number of unique in each column
bank_df.nunique()
Out[9]:
In [10]:
def df_col_analysis(df):
for name in df.columns:
print ("----------")
print ("Col Name: ", name)
print ("Col type: ", df[name].dtype)
if df[name].dtype is np.dtype('O'):
print (df[name].value_counts())
else:
print (df[name].describe())
df_col_analysis(bank_df)
In [11]:
bank_df.profile_report()
Out[11]:
In [12]:
cont_variables=['age','balance','day','duration','campaign','pdays','previous']
bool_vars=['default','housing','loan','target']
categorcial_variables = ['job', 'marital', 'education', 'default', 'loan', 'contact', 'month', 'day', 'poutcome']
for col in categorcial_variables:
plt.figure(figsize=(10,4))
sns.barplot(bank_df[col].value_counts().values, bank_df[col].value_counts().index)
plt.title(col)
plt.tight_layout()
In [13]:
def countplot(label, dataset):
plt.figure(figsize=(15,10))
ax=sns.countplot(x=label, data=dataset)
plt.show()
In [14]:
countplot("job", bank_df)
In [ ]:
In [15]:
bank_df['education'].value_counts()
Out[15]:
In [16]:
bank_df["Target"].value_counts()/bank_df["Target"].count()
Out[16]:
Target Analysis: An imbalanced dataset, as 88% of the observations are No as compared to just 12 % data as Yes.¶
In [17]:
# Another way to plot a histogram of duration is shown below
bank_df['duration'].hist(bins=50)
Out[17]:
In [18]:
plt.figure(figsize=(10,8))
sns.distplot(bank_df["age"])
Out[18]:
As we can see in the above distribution also, that most of the customers are in the age range of 30-40.¶
In [19]:
plt.figure(figsize=(10,8))
sns.distplot(bank_df["duration"])
plt.show()
This seems like a powerlaw distribution where most the values are very low and very few have high values.¶
In [ ]:
In [20]:
plt.figure(figsize=(10,8))
sns.distplot(bank_df["campaign"])
plt.show()
In [21]:
bank_df["pdays"].value_counts()
Out[21]:
Most of the values are -1, which means that the most of the customers have never been contacted before.¶
In [22]:
sns.boxplot(data=bank_df, x="Target", y="pdays")
plt.show()
In [23]:
plt.figure(figsize=(10,8))
sns.distplot(bank_df[bank_df["Target"]=="yes"]["pdays"])
sns.distplot(bank_df[bank_df["Target"]=="no"]["pdays"])
plt.show()
In [24]:
bank_df["previous"].value_counts()
Out[24]:
In [25]:
countplot("previous", bank_df)
In [26]:
bank_df["education"].value_counts()
Out[26]:
2. Multivariate analysis (8 marks)¶
- Bi-variate analysis between the predictor variables and target column. Comment on your findings in terms of their relationship and degree of relation if any. Visualize the analysis using boxplots and pair plots, histograms or density curves. Select the most appropriate attributes.
- Please provide comments in jupyter notebook regarding the steps you take and insights drawn from the plots
In [27]:
def barGraph(data, col1, col2 ):
msyes=data[col1][data[col2]=='yes'].value_counts()
msno=data[col1][data[col2]=='no'].value_counts()
ms=pd.concat([msyes,msno],axis=1)
ms.columns=['accepted','rejected']
print(ms)
ms.plot(kind='bar')
In [28]:
barGraph(bank_df, 'marital','Target')
single people are making more deposits them married and divorced.¶
In [29]:
barGraph(bank_df, 'education','Target')
Less educated people seems to get convinced by campaign¶
In [30]:
countplot("housing", bank_df)
Majority of the customers have a housing loan.¶
In [31]:
plt.figure(figsize=(10,8))
ax = sns.heatmap(bank_df.corr(),
annot=True,
linewidths=.5,
center=0,
cbar=False,
cmap="YlGnBu")
plt.title('Correlation')
plt.show()
In [32]:
bank_df.groupby(["job", "Target"])['job'].count()
Out[32]:
unemployed , self-employed and entrepreneur are not much inclined to subscribe for deposit¶
In [33]:
sns.pairplot(bank_df, hue = 'Target')
Out[33]:
In [34]:
sns.boxplot(data=bank_df, x="Target", y="duration")
plt.show()
Duration and Target: From the above plot it is clear that, the duration (last contact duration) of a customer can be useful for predicting the target variable. It is expected because it is already mentioned in the data overview that this field highely affects the target variable and should only be used for benchmark purposes.¶
In [35]:
sns.boxplot(data=bank_df, x="Target", y="campaign")
plt.show()
In [36]:
#People who converted were exposed to fewer campaigns than others
In [37]:
sns.boxplot(data=bank_df, x="Target", y="age")
plt.show()
Age :From the above boxplot we know that for both the customers that subscibed or didn't subscribe a term deposit, has a median age of around 38-40. And the boxplot for both the classes overlap quite a lot, which means that age isn't necessarily a good indicator for which customer will subscribe and which customer will not.¶
In [38]:
bank_df.groupby(["campaign", "Target"])['campaign'].count()
Out[38]:
People who converted were exposed to fewer campaigns than others¶
In [39]:
bank_df.groupby(["education", "Target"])['education'].count()
Out[39]:
In [40]:
bank_df.groupby(["month", "Target"])['month'].count()
Out[40]:
Months (Seasonal effect can been seen in the data. In may the sales increases)¶
Deliverable – 2 (Prepare the data for analytics) – (10)¶
- Ensure the attribute types are correct. If not, take appropriate actions.
- Get the data model ready.
- Transform the data i.e. scale / normalize if required
- Create the training set and test set in ratio of 70:30
In [41]:
#Categorize the column of dataset which is object type
for col in bank_df.columns:
if bank_df[col].dtype == object:
bank_df[col] = bank_df[col].astype('category')
In [ ]:
In [42]:
bank_df.isnull().sum()
Out[42]:
In [43]:
bank_df.info()
In [ ]:
Separate inpedendent and target variables¶
In [44]:
# values for "Target" : yes/no
bank_df["Target_cat"] = bank_df['Target'].map({'yes':1, 'no':0})
In [45]:
data_x = bank_df.iloc[:, :-2]
print("Shape of X:", data_x.shape)
data_y = bank_df["Target_cat"]
print("Shape of Y:", data_y.shape)
In [46]:
data_y.head()
Out[46]:
Transform the data i.e. scale / normalize if required¶
- Convert columns with 'yes' and 'no' values to boolean columns;
- Convert categorical columns into dummy variables.
- Remove not important columns
- Scale
In [47]:
#month can be category code values
data_x['month'] = data_x['month'].astype("category").cat.codes
In [48]:
# most of data of poutcome is unknown so it is not useful
data_x.drop('poutcome', axis=1, inplace=True)
In [49]:
# Drop 'contact', as every participant has been contacted.
data_x.drop('contact', axis=1, inplace=True)
In [50]:
#Map padys=-1 into a large value (10000 is used) to indicate
#that it is so far in the past that it has no effect
data_x.loc[data_x['pdays'] == -1, 'pdays'] = 10000
In [51]:
# values for "default" : yes/no
data_x['default_cat'] = data_x['default'].map( {'yes':1, 'no':0} ).astype("int")
data_x.drop('default', axis=1,inplace = True)
In [52]:
# values for "housing" : yes/no
data_x['housing_cat'] = data_x['housing'].map( {'yes':1, 'no':0} ).astype("int")
data_x.drop('housing', axis=1,inplace = True)
In [53]:
# values for "loan" : yes/no
data_x['loan_cat'] = data_x['loan'].map( {'yes':1, 'no':0} ).astype("int")
data_x.drop('loan', axis=1,inplace = True)
In [54]:
# Drop 'day' as they don't have any intrinsic meaning
data_x.drop('day', axis=1, inplace=True)
In [55]:
# Merge similar jobs into broad categories
data_x['job'] = data_x['job'].replace(['management', 'admin.'], 'white-collar')
data_x['job'] = data_x['job'].replace(['services','housemaid'], 'pink-collar')
data_x['job'] = data_x['job'].replace(['retired', 'student', 'unemployed', 'unknown'], 'other')
In [56]:
data_x.info()
In [57]:
# Convert categorical variables to dummies
data_x = pd.get_dummies(data=data_x, columns = ['job', 'marital', 'education' ], \
prefix = ['job', 'marital', 'education'])
data_x.head()
Out[57]:
In [58]:
data_x.shape
Out[58]:
In [59]:
data_x.info()
In [ ]:
In [60]:
X_train, X_test, y_train, y_test = train_test_split(data_x, data_y, test_size=.3, random_state=22)
X_train.shape, X_test.shape
Out[60]:
In [61]:
from sklearn.preprocessing import StandardScaler
sc = StandardScaler()
X_train= sc.fit_transform(X_train)
X_test= sc.fit_transform(X_test)
In [62]:
X_train.shape, X_test.shape
Out[62]:
Deliverable – 3 (create the ensemble model) – (30)¶
- First create models using Logistic Regression and Decision Tree algorithm. Note the model performance by using different matrices. Use confusion matrix to evaluate class level metrics i.e. Precision/Recall. Also reflect the accuracy and F1 score of the model. (10 marks)
In [63]:
## function to get confusion matrix in a proper format
def draw_cm( actual, predicted ):
cm = confusion_matrix( actual, predicted)
sns.heatmap(cm, annot=True, fmt='.2f', xticklabels = [0,1] , yticklabels = [0,1] )
plt.ylabel('Observed')
plt.xlabel('Predicted')
plt.show()
In [64]:
results_df = pd.DataFrame()
## function to get confusion matrix in a proper format
def print_metrics(results_df, testName, reg, actual, predicted ):
tr_acc=reg.score(X_train,y_train)
tst_acc=reg.score(X_test, y_test)
rc_scr = recall_score(actual,predicted);
pr_scr = precision_score(actual,predicted);
f1_scr= f1_score(actual,predicted)
roc_scr = roc_auc_score(actual,predicted);
print("Training accuracy",tr_acc)
print()
print("Testing accuracy",tst_acc)
print()
print('Confusion Matrix')
cm = confusion_matrix( actual, predicted)
sns.heatmap(cm, annot=True, fmt='.2f', xticklabels = [0,1] , yticklabels = [0,1] )
plt.ylabel('Observed')
plt.xlabel('Predicted')
plt.show()
print()
print("Recall:",rc_scr)
print()
print("Precision:",pr_scr)
print()
print("F1 Score:",f1_scr)
print()
print("Roc Auc Score:",roc_scr)
resultMap= {'1_TestName' : testName,'TrainingAccuracy' : tr_acc,
'TestingAccuracy' : tst_acc,'RecallScore': rc_scr,'PrecisionScore': pr_scr,'F1Score':f1_scr,
'ROCScore':roc_scr}
results_df=results_df.append(resultMap,ignore_index=True)
return results_df;
LogisticRegression¶
In [65]:
logreg = LogisticRegression()
logreg.fit(X_train, y_train)
y_pred = logreg.predict(X_test)
results_df= print_metrics(results_df,"LogisticRegression",logreg, y_test,y_pred)
In [66]:
# Running a loop to check different values of 'solver'
# all solver can be used with l2, only 'liblinear' and 'saga' works with both 'l1' and 'l2'
train_score=[]
test_score=[]
recall_score_list=[]
solver = ['newton-cg','lbfgs','liblinear','sag','saga']
for i in solver:
model = LogisticRegression(random_state=42,penalty='l2', C = 0.75,solver=i) # changing values of solver
model.fit(X_train, y_train)
y_predict = model.predict(X_test)
train_score.append(round(model.score(X_train, y_train),3))
test_score.append(round(model.score(X_test, y_test),3))
recall_score_list.append(round(recall_score(y_test,y_predict),3))
print(solver)
print()
print(train_score)
print()
print(test_score)
print()
print(recall_score_list)
recalls and other metrics are simillar . going to use default¶
Decision Tree¶
In [67]:
from sklearn.tree import DecisionTreeClassifier
In [68]:
#Using default 'gini' criteria to split. Other option include 'entropy'.
dTree = DecisionTreeClassifier(criterion = 'gini', random_state=1)
dTree.fit(X_train, y_train)
y_pred = dTree.predict(X_test)
In [69]:
results_df= print_metrics(results_df,"DecisionTreeClassifier",dTree, y_test,y_pred)
In [ ]:
In [ ]:
In [ ]:
2. Build the ensemble models (Bagging and Boosting) and note the model performance by using different matrices. Use same metrics as in above model. (at least 3 algorithms) (15 marks)¶
Ensemble Learning - Bagging¶
In [70]:
from sklearn.ensemble import BaggingClassifier
bgcl = BaggingClassifier(base_estimator=dTree, n_estimators=50,random_state=1)
#bgcl = BaggingClassifier(n_estimators=50,random_state=1)
bgcl = bgcl.fit(X_train, y_train)
y_predict = bgcl.predict(X_test)
results_df= print_metrics(results_df, "BaggingClassifier",bgcl, y_test,y_pred)
In [ ]:
In [ ]:
Ensemble Learning - AdaBoosting¶
In [71]:
from sklearn.ensemble import AdaBoostClassifier
abcl = AdaBoostClassifier(n_estimators=10, random_state=1)
#abcl = AdaBoostClassifier( n_estimators=50,random_state=1)
abcl = abcl.fit(X_train, y_train)
y_predict = abcl.predict(X_test)
In [72]:
results_df= print_metrics(results_df,"AdaBoostClassifier",abcl, y_test,y_pred)
In [ ]:
In [ ]:
Ensemble Learning - GradientBoost¶
In [73]:
from sklearn.ensemble import GradientBoostingClassifier
gbcl = GradientBoostingClassifier(n_estimators = 50,random_state=1)
gbcl = gbcl.fit(X_train, y_train)
y_predict = gbcl.predict(X_test)
In [74]:
results_df= print_metrics(results_df,"GradientBoostingClassifier",gbcl, y_test,y_pred)
In [ ]:
Ensemble RandomForest Classifier¶
In [75]:
from sklearn.ensemble import RandomForestClassifier
rfcl = RandomForestClassifier(n_estimators = 50, random_state=1,max_features=12)
rfcl = rfcl.fit(X_train, y_train)
y_predict = rfcl.predict(X_test)
In [76]:
results_df= print_metrics(results_df,"RandomForestClassifier",rfcl, y_test,y_pred)
3. Make a DataFrame to compare models and their metrics. Give conclusion regarding the best algorithm and your reason behind it. (5 marks)¶
In [77]:
results_df.T
Out[77]:
The campaign's goal is to have more people to subscribe term deposit i.e. less number of False Negative, so it don't lose customers who want to subscribe term deposit.
Types of error:
- Type I error — False postives : Predicted that Customer has subscribed term deposit but actually customer has not.
- Type II error — False negatives : Predicted that Customer has not subscribed term deposit but actually customer has.
In this use case, we need to reduce Type II error to not lose the people who would subscribe term deposits. Hence, Recall is MORE important then precision for our usecase.
Based on above metrics of various algorithms,
- LogisticRegression,BaggingClassifier, GradientBoostingClassifier , RandomForestClassifier have comparable accuracy of ~90 %
- LogisticRegression is simplest and fastest but recall is better for GradientBoostingClassifier and RandomForestClassifier of 0.42.
- Among RandomForestClassifier and GradientBoostingClassifier , we can pick either as they have comparable metrtics with GradientBoostingClassifier little better on accuracy.
In [ ]:
In [ ]: